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SUMMARY 

This  report  summarizes  the  results  of  four  experiments  on  digital  feature  data  base  requirements 
for  simulating  synthetic  aperture  radar  (SAR).  These  experiments  demonstrated  that,  a)  Digital 
Feature  Analysis  Data  (DFAD)  for  simulating  SAR  must  have  greater  density  than  the  current 
Defense  Mapping  Agency  (DMA)  products  Level  1  or  Level  2;  b)  the  primary  cues  for  SAR 
image  interpretation  are  lines  of  communication  and  large  natural  features;  c)  small  features 
must  be  depicted  in  the  DFAD  data  base  as  individual  reflectors  but  these  features  may  be 
depicted  generically  without  adversely  affecting  SAR  operator  task  performance,  and  d)  a  data 
base  with  reduced  individual  feature  fidelity  but  moderately  high  feature  density  will  support 
SAR  operator  task  performance  equally  as  well  as  a  high  feature  fidelity,  high  feature  density 
data  base.  These  results  indicate  that  feature  density  is  a  more  important  factor  than  feature 
fidelity  in  SAR  simulation.  We  therefore  recommend  that  to  support  SAR  simulation,  me  An 
Forcm  accept  the  newiy  developed  DMA  DFAD  specification,  Level  3-C,  which  incorporates  higher 
feature  density  than  Level  2  but  reduced  feature  fidelity. 
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SIMULATION  OF  SYNTHETIC  APERTURE  RADAR  IV: 
SUMMARY  AND  RECOMMENDATIONS 


I.  INTRODUCTION 

Simulating  Synthetic  Aperture  Radar  (SAR)  imagery  requires  Digital  Feature  Analysis  Data 
(DFAD)  with  greater  feature  density  than  is  currently  contained  in  standard  simulator  products 
from  the  Defense  Mapping  Agency  (DMA).  To  help  determine  DFAD  requirements  for  simulating 
SAR,  DMA  developed  a  prototype  high-resolution  DFAD  specification,  Level  X,  which  incorporated 
all  ground  features  which  were  10m  or  larger  on  a  side.  The  feature  density  in  Level  X  was 
much  greater  than  that  for  standard  products  such  as  Level  1,  which  has  a  100m  capture 
criterion,  or  Level  2  with  a  30m  capture  criterion. 

An  engineering  analysis  (TASC,  1985)  concluded  that  Level  X  would  support  SAR  simulation 
for  Weapon  System  Trainers  (WSTs).  Production  of  Level  X  in  quantity,  however,  is  not  practical 
due  to  the  coverage  required  by  the  using  major  commands  (MAJCOMs)  and  the  large  number 
of  man-hours  required  to  produce  Level  X  DFAD. 


II.  SUMMARY  OF  EXPERIMENTS 

Because  the  engineering  evaluation  of  Level  X  was  intended  only  to  determine  whether  Level 
X  was  adequate  for  simulating  SAR,  the  minimum  DFAD  requirements  for  simulating  SAR  were 
not  addressed.  Four  experiments  were  conducted  by  the  Operations  Training  Division  of  the 
Air  Force  Human  Resources  Laboratory  (AFHRL/OT),  in  cooperation  with  DMA  and  the  Aeronautical 
Systems  Division  (ASD/ENETV  and  ASD/YWB),  to  determine  minimum  simulation  requirements 
for  SAR.  These  experiments  are  briefly  summarized  in  this  section.  The  technical  details  of 
the  first  three  experiments  are  available  in  Crane,  Bell,  Kalinyak,  Dooley,  and  Hubbard  (1989); 
the  technical  details  of  the  final  experiment  are  reported  in  Crane  and  Bell  (1989). 


Experiment  1 


In  the  first  experiment,  SAR  simulations  were  generated  from  the  high-resolution  DFAD 
prototype,  Level  X,  with  a  10m  capture  criterion  and  from  a  current  DMA  product,  Level  2,  with 
a  30m  capture  criterion.  In  addition,  simulations  were  also  created  using  two  experimental 
DFADs  called  Levels  Y  and  Z  with  15m  and  20m  capture  criteria,  respectively.  These  simulations 
were  evaluated  by  SAR-experienced  Air  Force  officers.  The  simulations  generated  from  the 
current  DMA  product,  Level  2  (30m),  were  rated  not  acceptable.  The  subject-matter  experts 
also  indicated  that  the  10m  capture  criterion  in  Level  X  was  not  necessary  and  that  a  capture 
criterion  between  15m  and  20m  was  sufficient  for  acceptable  simulation  fidelity.  DMA  determined, 
however,  that  simply  reducing  the  number  of  features  included  in  the  DFAD  by  increasing  the 
capture  criterion  from  10m  to  15m  or  20m  would  not  increase  DFAD  production  sufficiently  to 
meet  Air  Force  needs. 


Experiment  2 


A  second  experiment  was  conducted  to  define  what  ground  features  are  critical  for  performing 
Radar  Scope  Interpretation  (RSI)  with  SAR  imagery.  SAR-experienced  Air  Force  officers  were 
asked  to  identify  those  features  within  aerial  photographs  which  they  would  use  to  help  them 
locate  specified  aimpoints.  The  features  most  often  identified  as  critical  were  lines  of 


communication  such  as  roads,  railroads,  and  canals,  and  large  natural  features  such  as  treelines, 
shorelines,  and  cultivated  fields.  Small  individual  features  such  as  structures  were  not  cited  as 
being  task-critical.  Nearly  all  of  the  critical  features  are  included  in  DMA  Level  2  DFAD. 

The  major  conclusion  drawn  from  Experiment  1  was  that  a  data  base  density  greater  than 
that  of  the  current  DMA  Level  2  DFAD  is  required  to  support  SAR  simulation.  The  results  of 
Experiment  2,  however,  indicated  that  RSI  for  SAR  is  based  primarily  on  the  large  structures, 
lines  of  communication,  and  natural  features  which  are  contained  in  Level  2  DFAD,  rather  than 
the  small  individual  features  unique  to  Levels  X,  Y,  and  Z.  Based  upon  the  comments  of  the 
subject-matter  experts  in  Experiments  1  and  2,  it  appears  that  the  small  individual  features 
contained  in  Levels  X,  Y,  and  Z  function  primarily  to  provide  the  ground  detail  needed  to  create 
a  SAR-like  appearance  within  the  simulated  SAR  image.  It  also  appears  that  regions  of  densely 
spaced  small  features  such  as  residential  areas  may  be  used  as  RSI  cues  without  reference  to 
individual  features.  If  this  is  indeed  the  case,  then  these  smaller  features  might  be  depicted 
generically  without  affecting  task  performance. 


Experiment  3 


The  hypothesis  that  small  features  such  as  houses  and  residential  streets  could  be  depicted 
generically  without  degrading  SAR  operator  task  performance  was  tested  in  Experiment  3.  B-1B 
Offensive  Systems  Officers  (OSOs)  performed  a  navigation  update  task  for  15  fixpoints  using 
simulated  and  actual  SAR  imagery.  The  SAR  simulations  were  generated  from  Level  X  (10m), 
from  Level  Y  (15m),  and  from  modified  versions  of  Level  2  (30m)  and  Level  1  (100m)  DFAD 
which  were  enhanced  with  generic  information  in  high-density  areas  (see  Crane  et  al.,  1989, 
for  an  explanation  of  the  enhancement  procedures).  In  addition,  photographs  of  actual  SAR 
imagery  were  obtained  for  each  fixpoint.  The  imagery  was  presented  to  the  OSOs  using  the 
Armstrong  Aerospace  Medical  Research  Laboratory’s  B-1B  Engineering  Research  Simulator  at 
the  338th  Combat  Crew  Training  Squadron,  Dyess  AFB,  Texas.  Placement  accuracy,  confidence 
in  placement,  and  rated  acceptability  for  mission  rehearsal  were  recorded.  The  results  showed 
that  levels  of  OSO  task  performance  for  enhanced  Level  2,  Level  Y,  or  Level  X  were  not 
significantly  different  from  each  other  or  from  photographed  SAR,  except  that  the  SAR  was 
rated  as  more  acceptable  for  mission  rehearsal  than  any  of  the  simulations.  Levels  of  placement 
accuracy,  confidence,  and  rated  acceptability  for  enhanced  Level  1  DFAD  were  significantly 
lower  than  for  other  DFAD  levels  and  actual  SAR. 

Taken  together,  these  three  experiments  indicate  that:  a)  High-density  data  are  required  to 
produce  simulations  which  have  the  appearance  of  SAR  images;  b)  these  high-density  data  do 
not  require  the  degree  of  ground  truth  typical  of  DMA  DFAD  for  features  smaller  that  those 
currently  included  in  Level  2  (30m)  DFAD;  and  c)  high-density  areas  can  be  added  to  Level  2 
data  through  the  use  of  texturing  and  the  breakup  of  larger  areal1  features  into  smaller  generic 
features  during  the  data  base  development  process.  Based  on  the  results  of  these  first  three 
experiments,  Crane  et  al.  (1989)  concluded  that  a  high-resolution  DFAD  product  should  be 
specified  with  high  accuracy  for  lines  of  communication  and  other  large  features,  and  that  less 
accurate  or  generic  information  would  be  acceptable  for  small  features  in  high-density  areas.2 


’An  areal  feature  is  distinguished  from  a  point  or  line  feature  in  that  it  is  a  two-dimensional  surface  delimited  by  a 
surrounding  border. 

2The  hypotheis  that  small  features  will  be  depicted  generically  presumes  that  if  the  aimpoint  is  a  small  feature  it  will  not 
be  depicted  generically  nor  will  small  features  be  generic  If  they  are  in  the  immediate  vicinity  of  the  aimpoint  and  likely  to  be 
used  as  RSI  cues.  In  these  cases,  higher  levels  of  ground  truth  will  be  required. 
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Experiment  4 


Although  the  first  three  experiments  led  to  a  definition  of  DFAD  requirements  for  simulating 
operational  SARs,  DMA  noted  that  current  resource  limitations  prevented  production  of  sufficient 
quantities  of  Level  2  DFAD.  Resources  for  generating  the  necessary  enhancements  were  also 
not  currently  available.  As  an  alternate  solution,  DMA  proposed  that  high-resolution  cartographic 
products  such  as  1:250,000  Joint  Operation  Graphics  and  1:50,000  topographic  maps  could 
serve  as  the  basis  for  a  high-resolution  DFAD.  The  prototype  description  of  this  product,  labeled 
Level  3-C  (cartographic),  indicated  that  it  would  have  a  data  base  density  similar  to  that  of  the 
enhanced  Level  2  DFAD  used  in  Experiment  3.  Compared  to  current  DFAD  products,  however, 
the  number  of  feature  codes  would  be  compressed  so  that  less  unique  information  regarding 
feature  type,  height,  or  reflectivity  would  be  available  for  each  feature.  Level  3-C  DFAD  would 
be  generated  using  existing  DMA  cartographic  production  facilities,  thereby  greatly  increasing 
the  availability  of  high-resolution  DFAD.  In  addition,  the  proposed  specification  would  be  flexible 
enough  to  allow  the  using  MAJCOMs  to  identify  critical  features  such  as  aimpoints  for  insertion 
into  the  DFAD. 

The  objective  of  the  fourth  and  final  experiment  in  this  series  was  to  evaluate  the  proposed 
Level  3-C  DFAD  for  use  in  SAR  simulations.  This  experiment  was  conducted  to  ensure  that 
simulated  SAR  produced  using  the  proposed  Level  3-C  DFAD  specifications  would  support  B-1B 
OSO  performance  as  well  as  did  the  Level  X  or  enhanced  Level  2  used  in  Experiment  3.  The 
detailed  description  of  this  experiment  is  contained  in  Crane  and  Bell  (1989). 

Data  bases  representing  nine  of  the  scenes  used  in  Experiment  3  were  developed  using 
DMA-supplied  prototype  Level  3-C  DFAD.  Data  bases  for  the  same  nine  aimpoints  were  also 
developed  using  the  enhanced  Level  2  and  Level  X  DFAD  from  Experiment  3.  Simulated  SAR 
imagery  was  then  generated  using  AFHRL’s  Advanced  Visual  Technology  System  (AVTS).  The 
details  of  the  AVTS  SAR  simulation  process  are  described  in  Ferguson  et  al.,  1989. 

The  resulting  SAR  images  were  transferred  to  the  B-1B  Engineering  Research  Simulators  at 
the  28th  Bomber  Wing,  Ellsworth  AFB,  South  Dakota,  and  the  319th  Bomber  Wing,  Grand  Forks 
AFB,  North  Dakota.  SAR-experienced  OSOs  performed  a  navigation  update  task  similar  to  that 
used  in  Experiment  3. 

Overall,  placement  accuracies  using  images  created  from  Level  X,  enhanced  Level  2,  actual 
SAR,  and  Level  3-C  were  not  significantly  different  from  each  other.  In  addition,  confidence 
ratings  and  ratings  of  acceptability  were  not  affected  by  the  level  of  DFAD  used  to  create  the 
simulator  data  base. 


III.  CONCLUSIONS 

This  series  of  experiments  evaluated  the  effects  of  DFAD  density  and  feature  accuracy  on 
the  quality  of  SAR  simulations.  The  data  from  these  experiments  indicate  that  major  lines  of 
communication  and  large  areal  features  are  critical  RSI  pointers  which  must  be  represented 
within  the  DFAD.  These  experiments  also  indicate  that  feature  density  must  be  greater  than 
current  Level  1  or  Level  2  DFAD  densities  for  acceptable  SAR  simulation.  The  feature  density 
of  the  prototype  Level  3-C  DFAD  is  sufficient  to  produce  an  acceptable  simulation  of  the  current 
B-1B  SAR.  In  addition,  the  compression  of  feature  identifiers  characteristic  of  Level  3-C  DFAD 
did  not  adversely  affect  performance.  This  indicates  that  feature  density  is  a  more  critical 
variable  than  feature  uniqueness  in  simulating  small  features  for  B-1B  SAR. 
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IV.  RECOMMENDATIONS 

The  following  recommendations  are  provided  regarding  Synthetic  Aperture  Radar  (SAR) 
simulation  requirements: 

1  That  the  Air  Force  accept  the  proposed  Level  3-C  as  a  high-resolution  Digital  Feature 
Analysis  Data  (DFAD)  product  to  support  SAR  simulation  for  the  B-lB  and  F-15E  Weapon  System 
Trainers. 

2.  That  DMA  produce  Level  3-C  DFAD  to  meet  current  SAR  simulation  requirements 

3.  That  DMA  and  the  Air  Force  begin  now  to  address  next-generation  sensor  simulation 
requirements. 
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